Running head: INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 1 Investigating the Dynamic Properties of Reward Processing: A Shift in Incentive Motivation Converts an Aversive Salt Cue into an Appetitive Motivational Magnet by

Increased diagnosis of disorders caused by abnormalities within reward circuitry, including addiction, illustrates the need to further define reward processing and enhance treatment. Focus has been placed on incentive salience or ‘wanting’, which dictates the value of a Pavlovian conditioned stimulus (CS) through associations with rewards (UCS), and converts a CS into a motivational magnet that energizes individuals’ effort to acquire rewards. We investigated whether incentive salience can be dynamically altered at salt CS re-encounter in the sodium depleted state and without re-experiencing the UCS in such state, to convert an aversive salt CS into an appetitive motivational magnet. Further, we activated the central amygdala (CeA) through μ opioid agonist DAMGO to measure the CeA’s role in dynamic incentive motivation. All testing was conducted in an autoshaping apparatus. The results illustrate that incentive salience can be rapidly altered, causing individuals to ‘want’ a CS they have never ‘liked’. INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 3 Investigating the Dynamic Properties of Reward Processing: A Shift in Incentive Motivation Converts an Aversive Salt Cue into an Appetitive Motivational Magnet The way that the human body processes rewards, specifically in terms of neural circuitry within the brain, has been a topic of interest in biopsychological research. The element driving current research is an attempt to understand the complexity surrounding psychological and neurobiological events that produce the pleasure associated with a reward (Berridge, Robinson, Aldridge, 2009). For instance, although most individuals will classify rewards as desirable and capable of producing a conscious awareness of pleasure, researchers have begun to show that this is not entirely the case (Berridge et al., 2009). Rather, what has become apparent is that the processing of rewards can occur in a conscious state as well as an unconscious state, without an individual’s awareness that they are experiencing a reward (Berridge et al., 2009). Further, it is believed that two underlying mechanisms, ‘liking’ and ‘wanting’ are at work during the processing of a reward, and can occur in a conscious or unconscious state. ‘Liking’ is an often unconscious, hedonic response, or pleasurable response to stimuli; whereas ‘wanting’ is a measure of the motivational value of stimuli, and can occur in conscious or unconscious state (Berridge, 2009a; Berridge et al., 2009). The ‘liking’ component of a reward therefore enables individuals to experience elements of their environment as pleasurable, and the ‘wanting’ component of reward enables these objective and often unconscious pleasures to be acquired (Berridge, 2009b). Accordingly, the two mechanisms underlying reward processing can be studied in conjunction, or can be targeted in specific ways in order to deeply understand the functionality of a single process. The main focus of this paper will be on the ‘wanting’ component of reward, also known as incentive salience. INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 4 Incentive salience is a form of desire that operates by its own set of deterministic rules, and has unique brain mechanisms for activation (Berridge, 2009a). The incentive salience hypothesis states that the motivational value, or ‘wanting’ of a Pavlovian conditioned stimulus, results from the interaction between the learned value of a reward and the current mesolimbic state of a subject relative to a reward. Accordingly, incentive salience can apply to innate incentive stimuli, such as a natural reward which serves the role of a Pavlovian unconditioned stimulus (UCS), or to learned stimuli that were originally neutral yet now predict a natural reward, and accordingly serve as a Pavlovian conditioned stimulus (CS) (Berridge et al., 2009). In the case of a learned reward, incentive salience becomes attributed to a CS by way of limbic mechanisms which draw upon the associations between CS and reward at the time of ‘wanting’, converting the shape, smell, or even sound of a CS into a captivating incentive (Berridge, 2009a; Berridge et al., 2009). In such a way, a CS becomes what researchers like to call a motivational magnet, or an object that energizes individuals’ behavior in an effort to actively seek and acquire a reward (Berridge et al., 2009). Two distinct types of CS exist, which each serve as different targets for reward responding: predictive CS are associated and occur in conjunction with reward, and contiguous CS are associated with the location of reward delivery. Interestingly, it has been shown that individuals display a tendency to direct their incentive salience towards specific stimuli in their environment. Previous research has found that incentive salience causes individuals to fixate on one type of learned CS: either the predictive CS associated with a reward, or the contiguous CS associated with reward delivery (Berridge et al., 2009). Accordingly, those with incentive salience directed towards the associated CS are deemed sign-trackers, whereas those with reward-specific CS are deemed goal-trackers (Berridge et al., 2009). This variation in behavior INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 5 further highlights the complexity of reward processing in the brain, as two individuals may not seek rewards in the same manner. The implications for this type of research become evident when considering the vast number of disorders that are apparently caused by abnormalities within the reward circuits of the brain. These include but are not limited to mania, substance dependence and abuse, and schizophrenia (Simon, 2010). It is believed that many reward disorders are caused by a sensitization of the incentive salience mechanism, which under normal circumstances is able to distinguish between competing rewards such as food, sex, or other types of pleasure; however under a sensitized state, causes the incentive salience mechanism to produce intense ‘wanting’ that is hyperspecific to certain stimuli (Berridge, 2009a). A specific example is drug addiction, which is believed to be a special case of intense ‘wanting’ to take drugs that causes individuals to become hypersensitive and reactive to stimuli that they have associated with drugs (Berridge, 2009a; Berridge et al., 2009). Crack cocaine addicts are known to “chase ghosts”, or continuously search for white granules that are similar in appearance to cocaine, even though the addicts know that these granules are not in fact cocaine (Berridge, 2009a; Berridge et al., 2009). Consequently, individuals with drug addictions often develop an intense ‘wanting’ to take drugs that eventually surpasses their ‘liking’ of a drug, causing individuals to seek out a drug more without actually experiencing the original pleasure they once felt from its use (Berridge, 2009a). The idea of a sensitized incentive salience can also be attributed to food addictions, although this research is still in its infancy and is largely believed to be a disorder of the ‘liking’ mechanism of reward (Berridge, 2009b). Regardless of the reward circuitry being targeted, it becomes apparent that knowledge regarding reward processing within the brain has the potential to change the way we understand and treat many disorders in our society. INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 6 Research has attempted to target incentive salience pathways in brain regions known to play critical roles in reward processing, including the central amygdala (CeA) (Mahler & Berridge, 2009). Other regions of interest include the nucleus accumbens and ventral pallidum (Mahler & Berridge, 2009). The amygdala is a brain structure where Pavlovian learning is translated into motivational salience, and it is believed that the opioid neurotransmission of the CeA is important in generating such incentive motivation (Mahler & Berridge, 2009). One specific study observed the impact of CeA microinjections of the μ opioid agonist DAMGO on incentive motivation behaviors associated with each individual’s prepotent cue, which is determined by their sign or goal-tracking behavior (Mahler & Berridge, 2009). The results of the study showed that opioid stimulation of the CeA intensified incentive salience towards rat’s prepotent reward cue, in terms of appetitive and consumatory behaviors, and accordingly made them ‘want’ the reward cue more; with the most predominant effect on sniffing and nibbling of the reward cue (Mahler & Berridge, 2009). These findings verify that there is a neural basis for incentive salience, which can be stimulated within specific regions of the brain using drugs that mimic the molecules released during reward. Aside from characterizing regions of the brain that underlie incentive salience behaviors, researchers have begun to more deeply explore our understanding of incentive salience by targeting and testing aspects of the proposed incentive salience hypothesis. The incentive salience hypothesis emphasizes the learned value of a reward as a fundamental aspect to the established motivation an individual experiences towards a reward. This statement implies that two processes must occur in order for an individual to possess incentive motivation to acquire a reward: an individual must learn the association between CS and UCS, which predict the presence of each other, as well as simultaneously learning the value of such UCS in order to INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 7 assess whether the UCS and thereby the CS is desired. For many years, researchers had emphasized the importance of learning in governing behavior towards CS and their associated UCS rewards. Specifically, it was believed that in order for an individual to desire a reward, they required previous experience in a motivational state that caused them to desire such reward; or in other words, individuals had to learn that they desired a reward in order for them to desire it once again (Dickinson & Dawson, 1988). However a developing question is if an individual must always use a method of re-learning in order to evaluate stimuli in their environment, or whether, when in unique states of hunger or thirst, re-learning can be adaptively disregarded in order to rapidly acquire a metabolically demanded stimulus in their environment. And of interest to reward processing, can individuals who have been depleted of a stimulus in their environment instantaneously ‘want’ such stimulus, even if they have never experienced such stimulus as ‘liked’? One study in particular attempted to answer this question by studying the rate of firing by neurons in the ventral pallidum, another structure believed to be involved in incentive salience processing (Tindell, Smith, Berridge, Aldridge, 2009). The study found that when rats were presented with a previously aversive salt CS in a now sodium depleted state, the incentive value of salt was enhanced, with neurons firing as vigorously as they did to an appetitive sucrose CS within the first CS presentation; all of which was shown in extinction of the actual UCS reward (Tindell et al., 2009). The fact that this testing condition was in extinction of the UCS means that the rats, up until this point, had not yet tasted the salt as a stimuli that they ‘liked’, yet still ‘wanted’ it exponentially more than in baseline tests (Tindell et al., 2009). Further, these results provide a more complete interpretation of incentive salience: that incentive salience is also influenced by current states of physiological appetite and satiety, and therefore is dynamically INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 8 altered at the moment of CS re-encounter so that a CS can be re-evaluated for its value; regardless of whether in fact the UCS is present (Tindell et al., 2009). An alternative way of describing the above results is in a learning-focused manner, in which latent learning is occurring. Latent learning is understood as an incidental acquisition of information from an individual’s environment that involves making associations between experiences with no immediate implication on an individual’s behavior; yet such knowledge can be recalled in a deprived state, in which individuals will act adaptively and utilize previously formed associations to manipulate their current behavior towards an environmental stimulus (Krieckhaus & Wolf, 1968; Stouffer & White, 2007). This theory of latent learning can be applied to studies like the one above, which utilizes sodium deprivation as a means of altering incentive salience. The results could therefore be viewed as an incidence in which rats are associating a prior experience of sodium ingestion, at a time when sodium was not needed, with their current need for sodium; thereby altering their behavior towards the salt CS (Krieckhaus & Wolf, 1968; Stouffer & White, 2007). Similar to incentive salience, latent learning may be also be influenced by contextual cues for internal motivational states, such as hunger, thirst, and sodium appetite, which allow individuals to recall stored information relevant to such a cue. This theory is further corroborated by research on maze tasks, in which variations in diet caused rats to change their direction in the maze towards the previously known location of a food or water reward (Hsaio & Isaacson, 1971; Stouffer & White, 2005). It becomes evident that the concepts set forth by the latent learning hypothesis are often complementary with the concepts set forth by the incentive salience hypothesis. However, there are rather subtle differences that distinguish the two ways of thought. Firstly, the arguments behind latent learning place emphasis on the ability of an individual to utilize previously formed INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 9 associations to alter their behavior towards a CS, thereby changing their current interpretation of a previously learned association (Krieckhaus & Wolf, 1968; Stouffer & White, 2007). Rather, the incentive salience hypothesis points to a change in the value of a UCS and thereby the CS as responsible for the changes observed in an individual’s behavior towards a CS; and accounts for attributes of a CS, including motivational magnet properties, which explain the shift in individual’s attractive behavior towards a CS (Berridge et. al, 2009). Further, studies measuring latent learning have often observed changes in behavior towards an appetitive CS in a single state, either causing an enhancement or decline in an individual’s behavior towards the CS (Krieckhaus & Wolf, 1968; Stouffer & White, 2007). Rather, studies measuring incentive salience have often observed whether behavior towards a previously aversive CS can be rapidly shifted from aversive to desirable (Berridge et. al, 2009). Despite the findings presented by the latent-learning hypothesis, and the more specific interpretations made by the incentive salience hypothesis, differing viewpoints continue to exist regarding the role of incentive salience in generating salt appetite when in a sodium depleted state. The primary alternative explanation that could be argued for the results presented in the Tindell article could be that inducing a sodium depletion triggers undirected activation towards both salt and sucrose CS rather than motivation for a specific CS (Berridge, 2001; Zhang et al., 2009). With the knowledge that there exists a debate between researchers regarding the role of incentive salience in mediating spontaneous appetitive behavior towards reward-related cues, it is evident that further research must be done in order to more deeply understand the specific motivation for and value attributed to various CS when in a UCS-depleted state. Accordingly, the goal of my study will be an attempt to produce results illustrating that incentive salience can be dynamically altered at the moment of CS re-encounter when in the INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 10 UCS-depleted state, so that a previously aversive CS can become a highly appetitive motivational magnet; without any re-tasting of the UCS in such state. Further, I will activate the CeA through the μ opioid agonist DAMGO, in order to measure CeA’s role in dynamic incentive motivation behaviors during reward encounter. Based on previous research illustrating that CeA μ opioid receptor activation caused an intensified incentive motivation towards individual’s prepotent cue, I hypothesize that CeA activation with DAMGO will intensify an individual’s desire for a previously aversive CS, most predominantly when in the depleted state, in comparison to CeA microinjections of a vehicle control. I will test this hypothesis through direct manipulation of the internal sodium ion concentration of rodent models. Sodium depletion is expected to transform an aversive salt stimulus into an attractive stimulus and therefore may warrant a motivated response, despite the fact that the stimulus has never been experienced as ‘liked’. Two experiments will be utilized within the study: one in which subjects receive no cranial microinjections, and one in which subjects receive microinjections of DAMGO and a vehicle control into the CeA. Accordingly, the goal of the first experiment will be to illustrate whether incentive shift is possible, whereas the goal of the second experiment will be to measure the influence of opioid receptor activation in the CeA, via DAMGO microinjection, on rat’s incentive shift towards a previously aversive salt cue. All testing will be conducted in an autoshaping apparatus, which is a form of Pavlovian conditioning that will enable individuals to learn associations between CS and UCS rewards. Method Subjects Female Sprague Dawley rats (n=20, 220-300 g), all bred in-house, were used in the study. Rats were housed in opaque, plastic cages containing wood shaving bedding, on a reverse 12 h INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 11 light/12 h dark cycle room; lights off at 9 am, lights on at 9 pm. 2-3 female occupants were housed in each cage, up until sodium depletion procedures when they were housed alone. Chow and water was available ad libitum, except 6-7 days post surgery and onward through autoshaping procedures, in which a restricted diet of roughly 15-20 g per day per rat was given. Materials Autoshaping testing apparatus and general procedure. Prior to training in the autoshaping apparatus, rats were handled for 5-7 days to adjust to human touch. Autoshaping chambers (30.5 x 24.1 x 21.0 cm) are comprised of two steel front and back plates, and clear plastic sides, floor, and ceiling (Mahler and Berridge, 2009). Strips of LED lights are mounted on the outside of the chamber in order to visualize behavior. A computerized program, MedPC, automatically controls the apparatus and runs autoshaping sessions. Four retractable levers are present in the autoshaping apparatus, and serve as the CS in the study. One lever is always extended, serving as a control for lever pressing, and is therefore classified as a CS-. The other three levers are extended periodically in conjunction with a reward during autoshaping sessions, and therefore serve as the CS+ in the study; one of the three periodically extended levers is dedicated to pellet autoshaping, whereas the other two are dedicated to infusion autoshaping. The MedPC program automatically scores any responding on the levers by the rats. There are multiple types of CS+ levers, dependent upon the testing conditions and the identity of the UCS; these include CS+ pellet levers, CS+ sucrose levers, and CS+ salt levers. Each of the CS+ levers are paired with visual and auditory recognition stimuli: a white LED light, which is illuminated during cue presentations, and a corresponding sound of clicking, white noise, or tone; the clicking is associated with the CS+ pellet lever, and the white noise and tone with the CS+ sucrose/salt levers. All auditory stimuli were standardized for an auditory INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 12 level of 80 dB, and were counterbalanced for sucrose/salt infusion trials to provide variation between the CS+ levers. Sucrose pellets, sucrose and salt infusions all assume the role of a UCS in the study, and accordingly there are two different types of autoshaping procedures. Pellet autoshaping. During sucrose pellet autoshaping, sucrose pellets are delivered through a sucrose pellet delivery cup located on the front, steel side of the apparatus, in-between the CS+ pellet and CS+ sucrose/salt infusion levers; as well as near the floor of the box so that it can be easily accessed by the rats. The delivery cup is equipped with a built-in infrared beam to measure the number of entries into the cup during CS+ pellet lever presentation, which lasts 8 seconds, as well as when the CS+ pellet lever is not extended. The intervals between CS+ pellet presentations were randomized between 30-90 s, and were automatically controlled by the MedPC program. Infusion autoshaping. During the second autoshaping procedure, infusion autoshaping, sucrose and salt infusions are delivered through thin tubing connected to the oral cannulae of the rats; which enters the apparatus through a small hole in the ceiling onto the rat’s head. The intervals between CS+ sucrose/salt presentations were randomized between 30-90 s, and were automatically controlled by the MedPC program. Following each CS+ presentation, a 0.11 mL quantity of sucrose/salt solution was infused for 6 s. During infusion autoshaping testing days, measurements were taken for two different testing conditions: in extinction of the UCS, followed by reinforcement with UCS. During the extinction trials, CS+ sucrose and CS+ salt presentations occurred in the absence of sucrose or salt infusion in order to establish the behavior of each rat towards the CS+ uniquely. During the reinforced trials, CS+ sucrose and CS+ salt presentations were followed by sucrose or salt infusion, similar to training. Extinction and INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 13 reinforced trials were maintained throughout test days for the purpose of determining whether a substance can be ‘wanted’ without ever having been ‘liked’. By utilizing an extinction trial, rats experienced autoshaping tasks without the presence of the solution, therefore disabling them from tasting the solution at that time and passing judgment on whether or not such substance was ‘liked’. Accordingly, what is shown during extinction trials is the incentive motivation towards a CS+ lever independent from any ‘liking’ or ‘disliking’ associated with the taste of the solution. Drugs and injections. A 2 μg /μl concentration of DAMGO (Sigma-Aldrich), dissolved in artificial cerebrospinal fluid (ACSF), was utilized for μ opioid activation in the CeA. DAMGO was administered through bilateral microinfusion into the cranial cannulae aimed at the CeA. ACSF was utilized as a control injection, also administered via cranial microinjection. Both a 7.5 mg/kg concentration of furosemide (Hospira) and a 5 mg/kg concentration of deoxycortisone acetate (Sigma-Aldrich) were utilized to induce sodium depletion in the rats. Both injections were administered subcutaneously. A combination of a 80 mg/kg concentration of ketamine (Fort Dodge Animal Hospital), a 7 mg/kg concentration of xylazine (Lloyd Laboratories), and a 0.04 mg/kg concentration of atropine (Neogen), were utilized during surgical procedures as anesthetics; all administered through intraperitoneal injection. Both 5 mg/kg of carprofen (Pfizer), a pain-killer, and 60 mg/kg of chlorampenicol (Sigma-Aldrich), an antibiotic, were given at the time of surgery, one day post-op, and on a need basis from then on; both administered through subcutaneous injection. Procedure Experimental design. During the study, subjects were divided into two different experiments. The subjects in the first experiment (n=4) underwent surgery to introduce only the oral cannulae to allow for sucrose and salt infusion during autoshaping testing. The subjects in INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 14 the second experiment (n=11) underwent surgery to introduce oral cannulae for sucrose/salt infusion as well as cranial cannulae aimed at the CeA for DAMGO microinjection during autoshaping testing. It is important to note that experimental procedure was altered between the first and second experiments in order to account for the addition of the cranial cannulae and therefore CeA stimulation in experiment two. Specifically, changes occurred in surgical procedure as well as training and testing conditions. Further, a within-subjects design was utilized throughout the experiment and during data analysis in order to account for the differences in experimental procedure. The specific details of each experimental procedure are outlined below. Experiment one. Phase one: Pellet autoshaping. Prior to infusion training, subjects underwent phase one of their training: 8 days of pellet autoshaping. In this setup, the presentation of the CS+ pellet lever, which lasted for 8 s, predicted the release of a single 45 mg sucrose pellet into the delivery cup. The CS+ pellet lever identity was the same for all autoshaping boxes, so that each animal was trained in the same manner; specifically, the lever to the left of the sucrose delivery cup served as the CS+ pellet lever, while the lever on the back left wall served as the CS-. The day before training began, the rats were given sucrose pellets in their cages. This pre-exposure to the pellets served to eliminate any neophobia towards the pellets. On the first training day, the rats underwent magazine training in which 25 UCS sucrose pellets were given in the absence of the associated CS+ sucrose lever, in order to familiarize them with retrieving pellets from the delivery cup (Mahler and Berridge, 2009). During this trial, the CSlever was extended continuously, however the CS+ pellet lever was never extended. For the next 7 days, sessions consisted of 25 pairings of the CS+ pellet lever and one UCS sucrose pellet delivery immediately INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 15 after retraction of the lever. On the 8 day of training, video recordings were taken and behavior was scored in order to establish sign or goal-tracking behavior. Specific behaviors that were observed were looks, sniffs/nibbles/licks, and slow bites. Surgery. Animals were anesthetized with ketamine (80 mg/kg), xylazine (7 mg/kg), and atropine (0.04 mg/kg), and surgically implanted with bilateral intraoral cannulae for sucrose and salt infusion (Mahler and Berridge, 2009). Carprofen (5 mg/kg), chlorampenicol (60 mg/kg), and 3 mL saline were also given at the time of surgery, and as necessary post-surgery. Oral cannulae were anchored to the skull with bone screws and cement. Phases 2-3: Sucrose/Salt infusion training. Animals were subjected to numerous conditions for sucrose/salt infusion training. All infusion training involved connecting salt or sucrose tubing to the oral cannulae of the rat, either unilaterally or bilaterally for infusion of solution, prior to placing the rat in the autoshaping apparatus. Each infusion session consisted of the pairing of a CS+ lever with either a salt or sucrose infusion, during which the retraction of the CS+ lever predicted an immediate infusion of solution through the oral cannulae into the mouth. Further, the identity of the CS+ salt and CS+ sucrose levers remained the same for each subject throughout the training and testing process. 17% (0.5 M) sucrose and 9% (1.5 M) salt solutions were utilized for infusions. Following recovery from surgery, phase two of training was conducted, which consisted of sucrose solution infusion training in which 25 CS+ sucrose presentations were paired with sucrose infusion through the oral cannulae. The subjects were trained for 5 days in phase two. Phase three of training consisted of block trials of sucrose versus salt infusion, in which a 10 CS+ sucrose session was followed by a 10 CS+ salt session. Both the CS+ sucrose and CS+ salt levers were unique in position and corresponding sound. The subjects were trained for 5 days in INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 16 phase 3. The order of sucrose and salt infusion sessions was counterbalanced amongst animals. Further, daily alternating between left and right cannulae for salt and/or sucrose infusion occurred, in order to eliminate any confounding variables due to side preference. Baseline behavioral measurement. Following this training period, a baseline measurement was taken for two different testing conditions: in extinction of the UCS, and reinforcement with UCS. Baseline testing conditions lasted one day and resembled those of phase three training, with modifications for extinction trials in which solution was not administered. Each testing session was video recorded and scored for behavioral output at a later date. Following baseline measurements, 1 additional day of training ensued in order to maintain behavior and reduce the effect of extinction. This additional training day resembled phase three training. Sodium appetite. Immediately following baseline testing, the rats underwent a sodium appetite test in which they were presented 5 drinking bottles: 2 filled with 45 mL of 3% NaCl solution, and 3 filled with 45 mL of distilled water. Measurements of fluid consumption were taken the next day in order to establish each rat’s affinity towards drinking NaCl solution versus distilled water. Sodium depletion. Removal of salt from the body and establishment of sodium appetite was induced by a combination of furosemide, a diuretic that promotes sodium loss and stimulates angiotensin II production, and deoxycortisone acetate, a mineralocorticoid hormone which mimics aldosterone elevation and therefore shuts down the internal signals to elevate sodium concentration (Mahler & Berridge, 2009). Further, a sodium-free pellet diet (TestDiet) accompanied by distilled water was maintained immediately following depletion procedures in order to prevent sodium repletion; rats were pre-exposed to this pellet diet a few days prior in INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 17 order to eliminate any neophobia towards the pellets. Both furosemide (7.5 mg/kg) and deoxycortisone acetate (5 mg/kg) were given subcutaneously shortly after the sucrose and salt infusion training period, one day post baseline testing. An additional injection of furosemide (7.5 mg/kg) was given 2 hours later. On the following day, a sodium-depleted measurement was taken once again for the two different testing conditions: in extinction of the UCS, and reinforcement with UCS. Sodium depleted testing conditions resembled those of baseline testing. Each testing session was video recorded and scored for behavioral output at a later date. Sodium repletion. Following depletion testing, the same low sodium diet was retained and water was replaced with the drinking bottles utilized to measure sodium appetite. One day after testing, the rats were removed from their low sodium diet, placed back on their normal chow diet, and their sodium appetite from the previous day was measured. For the next 5-6 days, sodium appetite measurements were taken in each of the bottles in order to establish each rat’s affinity towards the NaCl solution as well as the distilled water. On the fifth day post-testing, a sodium repleted measurement was taken in the two different testing conditions: in extinction of the UCS, and reinforced with UCS. Sodium repleted testing conditions resembled those of baseline testing. Each testing session was video recorded and scored for behavioral output at a later date. Experiment two. The procedure for experiment two was similar to that used during experiment one, with a few variations in order to account for the addition of the cranial component for microinjection into the CeA. The variations in procedure are outlined below. Surgery. In addition to the methods outlined for experiment one, subjects also received implantation of bilateral microinjection guide cannulae (14 mm long, positioned 2 mm above INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 18 CeA) for drug microinfusion. Stereotaxic placement coordinates for cranial cannulae were aimed at a specific site within the CeA, -2.4 AP; +/4.0 ML; and -5.8 DV (mm relative to bregma) (Paxinos and Watson, 2007). Steel stylets were inserted into cranial cannulae to prevent entrance of foreign substances into the brain. Phases 2-4: Sucrose/Salt infusion training. In addition to phase two and phase three infusion training sessions, a phase four training session was introduced. Phase four of training consisted of a sucrose and salt dual infusion, in which 10 CS+ sucrose and 10 CS+ salt presentations occurred within the same session, totaling to 20 CS+ presentations; with only one solution being delivered per CS+. This was accomplished through the use of both of the oral cannulae, one corresponding to sucrose and one corresponding to salt. CS+ sucrose and CS+ salt presentations were administered in random fashion by the MedPC program. Additionally, variation occurred in the number of training days. Specifically, the subjects were trained for 5 days in phase two, for 3 days in phase three, and for 3 days in phase four. Baseline behavioral measurement. For the subjects in experiment two, baseline testing conditions resembled those of phase four training, with modifications for extinction trials in which solution was not administered. Variation also occurred in the amount of additional days of training between baseline and sodium depleted testing days; specifically, two additional days of phase four training occurred post-baseline testing, prior to sodium depletion testing. Drug microinjection. Each rat in experiment two received a mock-injection one day prior to baseline measurements, in order to acquaint them with the microinjection procedure. All rats received vehicle injections of artificial cerebrospinal fluid (ACSF) at this time. During drug microinfusion trials, baseline measurements were taken over a two-day period, in order to provide one day for drug and one day for vehicle treatment. Accordingly, the rats each received INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 19 an extinction trial with and without drug, but only a reinforced trial for their second day of testing; whether this be drug or vehicle treatment. This prevented the rats from tasting solutions prior to their second extinction test, thereby ensuring that the extinction tests all occurred without experiencing the solutions at that time in their given state. During the first day of baseline measurements, half of the subjects received 0.5 μL DAMGO microinjection, while half received a 0.5 μL injection of ACSF. On the second day of baseline measurements, the subjects who previously received DAMGO received ACSF, and vice-versa. Intracranial microinfusions into the CeA were administered prior to autoshaping testing. Infusions were given at a rate of 0.25 μL/min and lasted 2 minutes, with an additional minute allotted to allow for diffusion into the brain tissue. Following infusion, stylets were reinserted into the cannulae, and testing ensued. Sodium depletion. Sodium depletion was induced after the second day of additional training post-baseline testing. Sodium depletion testing conditions resembled those of baseline testing. Further, the subjects received drug and vehicle microinjections in the same manner as administered during baseline testing, causing sodium repletion testing to last two days. Sodium repletion. Sodium repletion testing conditions resembled those of baseline testing. Further, the subjects received drug and vehicle microinjections in the same manner as administered during baseline and sodium depleted testing, causing sodium-repletion testing to last two days. Behavioral recording and scoring. All behavioral output from experiment one and experiment two, including pellet training and infusion testing trials, was recorded by two cameras. One camera was positioned below the apparatus facing the transparent floor of the box, in order to observe the rat’s head and body in relation to the levers; and the other was positioned INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 20 to face the far side of the apparatus, perpendicular to the lever walls and pellet delivery cup, in order to observe interaction with the CS+ pellet lever as well as the pellet delivery cup. Pellet autoshaping. Although the pellet autoshaping program itself recorded lever pressing as well as entries into the food cup, alternative behaviors such as looks, sniffs/nibbles/licks, and slow bites were scored for the final day of pellet autoshaping via the side cameras to establish a rat’s sign-tracking or goal-tracking nature. Sign-trackers exhibited these behaviors predominantly with the CS+ pellet lever, whereas goal-trackers exhibited these behaviors predominantly with the sucrose pellet delivery cup. Of the 25 total CS+ pellet presentations, only the 5, 10, 15, 20, and 25 were scored for behavioral responses. Infusion autoshaping. Bottom camera observations were utilized to establish rat’s behavior during infusion autoshaping testing towards a specific lever, which predicted either sucrose or salt infusion. All CS+ sucrose and CS+ salt presentations were scored during the 8 seconds pre-CS+, the 8 seconds of CS+, and the 8 seconds post-CS+ presentation. Behaviors such as orientation, distance, approaches or avoidances, seconds in contact, latency to initiate an approach, latency to contact, looking, and rearing were observed and recorded. Orientation involved estimating the rat’s average orientation towards the lever on a scale of 0 to 180 degrees, based on the placement of their paws. Distance from the lever was estimated using labeled markings on the floor of the apparatus that noted 5, 10, 15, 20, 25 and 30 cm away from the lever; measurements were taken from the closest point on the rat’s body to the lever, not including the tail. Any movement of the paws towards the lever was categorized as an approach, or away from the lever was categorized as an avoidance; integer values were used, and continuous movement was scored as a single approach or avoidance. Seconds in contact was categorized as the amount of time, in seconds, that a rats body was within 1 cm of the lever, or INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 21 during non-CS+ presentations, the time that the rat would have been in contact had the lever been extended. Latency to initiate an approach was scored by noting the time the lever began to extend and the point at which an approach towards the lever was initiated, whereas latency to contact was scored by noting the time at which the rat came into contact with the lever in comparison to the time the lever began to extend; time was in seconds and milliseconds. Looking was scored as any glance directly at the lever or pellet delivery cup after being extended, or at the slot during non-CS+ periods, and was an integer value. Rearing was categorized as any movement in which the rats front paws were extended upwards and their back paws were firmly planted on the floor, once again scored in integer values. In addition, appetitive nibbles/bites, paw/lip licks, sniffing, and aversive gaping and flailing behaviors in relation to a CS+ lever were also observed and recorded. Nibbles/bites were scored as either a single biting motion at one side of the lever, or a swift movement across the lever from one end to the other; both behaviors were scored in integer values. Paw and lip licks were any distinct licks of the paws, which were complete and did not include stops or adjustments, and any visible movement of the tongue onto the lips in a lateral protrusion; both were scored in integer values. Sniffing was categorized as any interaction with or close proximity towards the lever for a brief period of time, without any extended movement across the lever; also scored in integer values. Gaping was scored as an opening of the mouth, often to the point of showing the lower teeth, in which the rat attempted to expel the fluid from their mouth, and was scored in integer values. Forearm flails and headshakes were characterized by a brief shaking of the hands or head at a fast pace, and were also integer values. Statistical analysis. In order to assess whether the autoshaping learning paradigm was successful, paired t-tests were utilized to compare appetitive behaviors towards CS+ salt v. INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 22 sucrose levers, appetitive behaviors during the 8 s duration pre-CS+ sucrose period v. the 8 s CS+ sucrose presentation, as well as aversive behaviors during the 8 s duration pre-CS+ salt period v. the 8 s CS+ salt presentation. All other results were analyzed by a repeated measures test and a one-way ANOVA with Tukey post-hoc measurements, which enabled analysis for multiple factors including testing day (baseline v. sodium depleted v. sodium repleted) as well as CS+ type (sucrose v. salt). A within subjects analysis was used. Additionally, results from the autoshaping test days for experiment two were analyzed in the same manner as above, however with the addition of a treatment factor (drug v. vehicle) which was analyzed alongside testing day and CS+ type. Histology. At the completion of sodium repletion testing for experiment two, rats were given an overdose of pentobarbital and decapitated. Brains were removed from rats that were implanted with cranial cannulae, sliced in 60 μm coronal sections, and stained with cresyl violet. Slices were observed under a microscope to verify cranial cannulae placement within the CeA. Results Experiment One For experiment one, one rat was excluded based on its inability to learn within the autoshaping apparatus, which was attributed to a possible blindness. An initial comparison was made between sodium appetite in baseline and sodium depleted states in order to assess whether a change in consumption of NaCl solution occurred. These results would illustrate whether the sodium depletion procedure successfully induced sodium appetite, and whether any shifts in incentive salience could be attributed to a change in appetite towards salt. It was observed that sodium appetite increased from baseline to sodium depleted testing days (t(4)=3.19, p=0.03), and INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 23 decreased back to baseline levels during the sodium repleted testing day (t(4)=0.19, p=0.86) (See Figure 1). Following the analysis on sodium appetite, the autoshaping procedure was analyzed to establish whether subjects were in fact learning associations between CS+ sucrose and CS+ salt levers and their associated UCS reward. The first test involved comparing appetitive behaviors during pre-CS+ and CS+ sucrose presentations of baseline tests in extinction of the UCS, in order to determine whether appetitive behaviors were occurring uniquely during the CS+ sucrose presentations, or rather throughout testing trials; all of which occurring without impact from tasting the UCS. Appetitive behaviors were greater during the 8 s CS+ presentations as opposed to the pre-CS+ sucrose period (t(3)=8.19, p<0.01) (See Figure 2). Next, a comparison between aversive behaviors during pre-CS+ salt and CS+ salt presentations of baseline tests in extinction was made, in order to determine whether aversive behaviors towards salt were occurring uniquely during the CS+ salt presentations. There was no difference in aversive behaviors during the CS+ salt presentations and the pre-CS+ salt periods (t(3)=2.29, p=0.11) (See Figure 3). This result would suggest that rats were not in fact autoshaping towards the CS+ salt lever. In order to further assess the validity of this finding, a comparison was made between aversive behaviors during pre-CS+ salt and CS+ salt presentations of baseline tests during reinforced trials. A greater amount of aversive behaviors occurred during the CS+ salt presentations as opposed to the pre-CS+ salt period (t(3)=3.61, p=0.04), illustrating autoshaping towards the CS+ salt lever (See Figure 3). Further, in order to assess whether a preference towards CS+ sucrose v. CS+ salt levers existed, salt and sucrose appetitive behaviors during baseline tests in extinction were compared. INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 24 It was found that a preference in appetitive behavior towards the CS+ sucrose lever during baseline extinction tests existed (t(3)=5.23, p=0.01). Following analysis on the autoshaping technique, appetitive behaviors were analyzed in order to assess whether a shift in incentive motivation towards CS+ salt levers occurred when in the sodium depleted state. An interaction (linear) was found to exist between CS+ type and testing day for appetitive behaviors towards CS+ levers (F(1,3)=53.02, p=0.01), with a significant difference between testing days for the salt solution in extinction trials (F(2,9)=5.30, p=0.03) but not for the sucrose solution in extinction trials (F(2,9)=0.50, p=0.62). With the knowledge that there was no interaction between sucrose and testing day but that instead there was an interaction between salt and testing day, comparisons were made between appetitive behaviors towards CS+ salt levers on baseline, sodium depleted, and sodium repleted testing days. First, appetitive behaviors during baseline CS+ salt tests in extinction were compared to sodium depleted CS+ salt tests in extinction in order to see if an increase in appetitive behaviors occurred in the depleted state. As expected, based on the induced sodium appetite, greater appetitive behaviors were found to occur during sodium depleted as opposed to baseline salt extinction tests (p=0.03) (See Figure 4). Next, a comparison was made between baseline and sodium repleted CS+ salt appetitive behaviors in extinction, in order to see if incentive motivation decreased during sodium repleted testing days back to baseline levels. Appetitive behaviors towards the CS+ salt lever during sodium repleted tests were no different from baseline (p=0.08) (See Figure 4). However, an interesting finding was observed when comparing sodium depleted to sodium repleted CS+ salt appetitive behaviors in extinction. It was found that appetitive behaviors towards the CS+ salt levers during sodium repleted tests were also no different from sodium depleted tests (p=0.85) INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 25 (See Figure 4). This finding illustrates that during extinction trials of repleted tests, the appetitive behavior of rats does not fully decrease to baseline levels. In order to assess whether tasting the solutions altered the patterns viewed during extinction trials, specifically in the repleted state, appetitive behaviors during reinforced trials were examined for all three testing days. An interaction (quadratic) was found to exist between testing days for appetitive behaviors towards CS+ levers in reinforced trials (F(1,3)=11.95, p=0.04), with a significant difference between testing days for the salt solution in reinforced trials (F(2,9)=13.74, p<0.01) but not for the sucrose solution in reinforced trials (F(2,9)=0.15, p=0.87). Specifically, appetitive behaviors towards salt increased from baseline to sodium depleted reinforced tests (p<0.01), and decreased from sodium depleted levels (p=0.01) to baseline levels (p=0.50) during sodium repleted tests (See Figure 5). This finding illustrates that only during reinforced trials of repleted testing, after having tasted the salt solution, do rats decrease their appetitive behavior towards the CS+ salt lever to baseline levels. In order to further understand the incentive shift, a comparison between sodium depleted CS+ salt v. sucrose appetitive behaviors during extinction tests was made, in order to see if the shift in incentive motivation towards salt quantitatively matched that of sucrose. It was found that CS+ salt appetitive behaviors were no different from CS+ sucrose appetitive behaviors in sodium depletion extinction tests (t(3)=1.49, p=0.23), and therefore quantitatively matched that of sucrose. A comparison was also made between appetitive behaviors during sodium depleted CS+ salt extinction and sodium depleted CS+ salt reinforced tests, in order to assess whether tasting the salt solution enhanced the ‘wanting’ of the associated CS+ salt lever. It was found that tasting the solution during the reinforced tests increased appetitive behaviors towards the INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 26 CS+ salt lever as opposed to extinction trials of sodium depleted CS+ salt tests (t(3)=3.92, p=0.03). Following analysis on the shift in incentive motivation, individual behaviors were analyzed for interactions between CS+ type (salt v. sucrose) and testing day (baseline v. sodium depleted v. sodium repleted) in order to assess which specific behaviors were correlated with the incentive motivation shift. An interaction (linear) between CS+ type and testing day existed for seconds in contact with CS+ levers (F(1,3)=31.69, p=0.01). Greater seconds in were spent in contact with the CS+ salt lever during sodium depleted as opposed to baseline extinction tests (t(3)=3.71, p=0.03). Further analysis revealed that rats were spending an increased amount of time with the lever during baseline CS+ sucrose opposed to baseline CS+ salt extinction tests (t(3)=5.10, p=0.02). However during sodium depleted extinction tests, rats increased their time spent with the salt lever and were spending an amount of time with the salt that was no different from sucrose CS+ levers (t(3)=1.12, p=0.34) (See Figure 6). An effect of CS+ type was also found for orientation towards CS+ levers (F(1,3)=15.79, p=0.03). Analysis revealed that during baseline salt tests in extinction, the angle of a rat’s body in relation to the CS+ salt lever was greater than that of baseline CS+ sucrose tests (t(3)=4.00, p=0.03). However during sodium depleted extinction tests, rats decreased their angle of orientation towards the CS+ salt lever and were angle no different from the CS+ sucrose lever (t(3)=2.44, p=0.09) (See Figure 7). Experiment Two For experiment two, three rats were excluded based on histology reports illustrating imprecise cranial cannulae targets (See Figure 8). To begin, the success of the sodium depletion procedure in inducing sodium appetite was assessed. Comparisons between sodium appetite in INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 27 baseline and sodium depleted states were made in order to assess whether a change in consumption of NaCl solution occurred. It was shown that sodium appetite was no different on baseline and sodium depleted testing days (t(10)=2.00, p=0.07), and was also no different from baseline (t(10)=0.15, p=0.89) as well as sodium depleted (t(10)=2.14, p=0.06) levels during the repleted testing day (See Figure 1). Further analysis on whether subjects were in fact learning associations between CS+ sucrose and CS+ salt levers and their associated UCS reward were made. Comparisons between appetitive behaviors during pre-CS+ and CS+ sucrose presentations of baseline tests in extinction, and between aversive behaviors during pre-CS+ salt and CS+ salt presentations of baseline tests in extinction were made in order to determine whether behaviors were occurring uniquely during the CS+ presentations, or rather throughout testing trials for each CS+ type. It was found that appetitive behaviors were greater during the 8 s CS+ presentations as opposed to the pre-CS+ sucrose period (t(8)=5.86, p<0.01). However, it was found that aversive behaviors during the CS+ salt presentations and the pre-CS+ salt periods were shown to be no different (t(8)=0.47, p=0.65). From these results, it appears as though autoshaping to sucrose but not to salt CS+ levers occurred. Following analysis on the autoshaping technique, behaviors were analyzed in order to assess whether a shift in incentive motivation towards CS+ salt levers was absent when in the sodium depleted state for drug and vehicle trials; due to a lack of sodium appetite and autoshaping towards the CS+ salt lever. It was found that an effect of CS+ type for appetitive behaviors in extinction (F(1,8)=25.75, p<0.01) existed, but there was no effect of drug treatment (F(1,8)=5.01, p=0.88) or testing day (F(1,8)=4.14, p=0.08). This finding suggests that drug v. vehicle treatment did not cause variation in behavior towards the CS+ levers, and rather the INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 28 identity of the CS+ lever influenced behaviors. Accordingly, behaviors were analyzed independently for vehicle and drug trials, so that only the comparison between CS+ types on the same testing day were made. Firstly, behaviors were observed in vehicle trials, in order to assess whether incentive shifts towards salt occurred regardless of the presence of drug. A comparison was made between baseline CS+ salt and CS+ sucrose appetitive behaviors, which revealed increased appetitive responses towards the CS+ sucrose lever (t(8)=3.76, p=0.01). An additional comparison between sodium depleted CS+ salt and CS+ sucrose appetitive behaviors during extinction trials was made in order to see if a shift in incentive motivation towards salt occurred that quantitatively matched that of sucrose. Appetitive behaviors towards CS+ salt levers did not quantitatively resemble those of CS+ sucrose levers during sodium depletion extinction trials, and rather appetitive behaviors towards CS+ sucrose levers were greater (t(8)=3.93, p<0.01) (See Figure 9). Based on these results, it is evident that an incentive shift did not occur. Behaviors were then observed under drug conditions, in order to see if DAMGO treatment produced a similar effect on the incentive shift towards the CS+ salt levers as did the vehicle treatment; seeing as there was no incentive salience towards salt to intensify. It was found that CS+ salt and CS+ sucrose appetitive behaviors were no different during baseline tests (t(8)=1.96, p=0.09), yet diverged during sodium depleted extinction tests and did not quantitatively match, with greater appetitive behavior towards the CS+ sucrose levers (t(8)=4.72, p<0.01) (See Figure 10). This finding is slightly different from the vehicle trial results, and illustrates a possible role for DAMGO as enhancing appetitive responses specific to sucrose. Discussion INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 29 Based on the results for experiment one, it is evident that an induced sodium depleted state within rats altered their behavior towards the CS+ salt lever, shifting it from aversive to appetitive. This increase in appetitive behavior towards salt illustrates that an alteration in rat’s motivation occurred, and made them want the salt stimulus more in the sodium depleted state. Further, it was found that the appetitive behaviors towards salt were no different from appetitive behaviors towards sucrose, which remained the same throughout testing days. These results not only illustrate that inducing a sodium depletion state did not affect rat’s motivation towards sucrose, but also that this state caused rats to ‘want’ salt just as much as they ‘want’ sucrose. Also, when comparing sodium depleted salt testing days in extinction of and reinforced with solution, it was observed that appetitive behaviors were increased during reinforced trials. The fact that reinforced conditions enhanced appetitive behaviors suggests that tasting the solution invoked ‘liking’ reactions towards the lever, which in combination with ‘wanting’ reactions led to a heightened appetitive response. This finding not only illustrates that tasting the solution enhanced ‘liking’ reactions and that the two processes influence each other, but also further supports the notion that ‘liking’ and ‘wanting’ components of reward can be parsed apart and tested for individually. An interesting finding was also observed when comparing sodium repleted salt testing days in extinction and reinforced trials. It was found that during extinction trials, appetitive behaviors did not completely decrease to baseline levels, however during reinforced trials, appetitive behaviors were no different from baseline levels. This finding suggests that although rats can ‘want’ a stimulus they have never ‘liked’, they must ‘dislike’ a previously ‘liked’ stimulus in order to no longer ‘want’ it. INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 30 In regard to specific behaviors that were influenced by the incentive shift, other than appetitive behaviors, it was found that motivation towards the CS+ salt lever presented itself in behaviors including increased seconds in contact with the lever, as well as a decreased angle of orientation from the lever. These two behaviors illustrate an apparent attraction towards the lever, causing rats to position themselves closer to and angled towards the lever. Analysis on the accuracy of the sodium depletion procedure in inducing sodium appetite and the autoshaping procedure in creating associations between CS and their associated UCS was conducted in order to validate the above results. In was first shown that inducing a sodium depleted state within rats successfully altered NaCl solution consumption from baseline to sodium depleted states, thereby inducing a sodium appetite. Therefore, the above results illustrating a successful incentive shift towards the salt CS+ lever can be viewed as a direct consequence of the sodium depletion technique. Further, it was evident that rats were in fact learning associations between CS+ levers and their associated solutions due to their increased behavior during the actual CS+ presentations as opposed to the pre-CS+ periods. Specifically, the fact that there was variation in behaviors towards the CS+ sucrose and CS+ salt levers, with increased appetitive behaviors towards the sucrose and increased aversive behaviors towards the salt levers, established a clear distinction in baseline behaviors towards solutions by which a shift could be observed. Although aversive behaviors towards salt were only present in reinforced and not extinction trials, there remains an argument for the learned aversive behavior towards the CS+ salt lever. This is most likely due to a limitation by which behaviors are scored, with a tendency to account for the presence of behaviors and not the absence of behaviors. It became apparent during video scoring of the baseline salt extinction trials that many rats remained motionless and INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 31 did not respond to the CS+ salt presentations in any manner. Therefore, although they were not displaying appetitive responses towards the lever, they also were not displaying any of the specific aversive responses being scored. Further, it is also possible that there was in fact an effect in extinction trials, however the low sample size created a large standard error, which thereby rendered the results not significant. Overall, the results from experiment one suggest that the proposed hypothesis was in fact correct. The results illustrate that incentive salience can be dynamically altered at the moment of CS re-encounter when in the sodium depleted state. Further, such dynamic alteration in incentive salience occurs without re-tasting the UCS in the sodium depleted state. And lastly, incentive salience is capable of converting a previously aversive CS into a highly appetitive motivational magnet. These findings corroborate previous research showing instantaneous incentive salience for salt when in the sodium depleted state (Tindell et al., 2009). However, these results go further to illustrate that alternative explanations for the results, such as undirected activation towards both salt and sucrose CS, did not occur and rather motivation for specifically the salt CS+ lever increased; with motivation for sucrose being largely unaffected (Berridge, 2001; Zhang, et al., 2009). The results from experiment two were not as conclusive as those from experiment one. Analysis on the accuracy of the sodium depletion procedure in inducing sodium appetite and the autoshaping procedure in creating associations between the CS and their associated UCS showed that inducing a sodium depleted state within rats may not have worked successfully; as a failure to alter NaCl solution consumption and therefore introduce a sodium appetite occurred. Further, it was evident that rats were in fact learning associations between CS+ sucrose levers and their associated solution, with increased appetitive behaviors. However, there did not appear to be a INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 32 similar learning towards the CS+ salt lever, which would have presented itself with increased aversive behaviors during the CS+ presentation period. Without a baseline aversive response towards the salt levers, a clear distinction could not be established for a potential shift in behavior during the sodium depleted state. In terms of shifting incentive salience towards salt, it was found that this did not occur and that rats did not alter their behavior towards the CS+ levers. Rather, during vehicle trials, rats illustrated low levels of appetitive behaviors to salt on both baseline to sodium depleted testing days, values that were significantly lower than sucrose appetitive behaviors on the same testing days. This lack of change in appetitive behavior towards salt illustrates that a change in rat’s incentive motivation did not occur, and therefore their ‘wanting’ for the salt lever did not increase in the sodium depleted state. Further, these findings illustrate that rats, under this testing condition, were not able to ‘want’ a stimulus they had never ‘liked’. Additionally, during drug trials, it was shown that DAMGO did not intensify incentive salience towards salt. This finding was expected, as rats did not show a general incentive motivation towards the salt CS+ lever, and therefore there was no incentive salience to intensify. Interestingly, it was shown that DAMGO treatment caused sucrose but not salt appetitive behaviors to increase from baseline to sodium depleted testing days, suggesting a possible role for DAMGO in enhancing incentive salience towards the appetitive sucrose CS+ lever. However, it is important to note that the appetitive responses towards sucrose in sodium depleted drug trials were no different from sodium depleted vehicle trials, and therefore the effect of drug in comparison to vehicle can be viewed as minimal. The results from experiment two do not support the proposed hypothesis based on previous research showing that opioid stimulation of the CeA intensified incentive salience INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 33 towards rat’s prepotent reward cue (Mahler & Berridge, 2009). The lack of sodium appetite alongside the failure to associate the CS+ lever with the aversive salt solution may be the reasons why the majority of rats did not successfully shift their incentive motivation towards the salt levers. Specifically, the lack of sodium appetite may have been due to a failure of the drug combination to induce sodium depletion, imprecise drug administration, or other endogenous factors. Rather, the drugs may have induced a general thirst instead of a sodium appetite. The possible role for increased thirst is further corroborated by increased behavior towards the sucrose levers occurred during drug trials, illustrating a general ‘want’ for solution to relieve the dehydrated state, and therefore a preferential response towards the appetitive sucrose levers. Despite the fact that experiment two did not yield the results that were hypothesized, fundamental conclusions regarding incentive salience can still be drawn from the results that were obtained. Based on the knowledge gained from experiment one, incentive salience can now be understood as a dynamic property that can be instantaneously altered when in the UCSdepleted state, without requiring any re-tasting of the UCS in such state. Further, it was shown that a previously aversive CS could be converted into a highly attractive motivational magnet that would warrant an appetitive response when in the UCS-depleted state. These two results corroborate the hypothesis for experiment one, and support the notion that incentive salience can be rapidly altered to acquire a demanded stimulus from the environment, even if such stimulus has never been ‘liked’ or desired before. Although results were obtained from the study, there appear to be some limitations in the experimental design. One apparent weakness presents itself in the method by which some behaviors are accounted for. As previously mentioned, a tendency to account for behaviors rather than the lack of behaviors is a minor flaw in the scoring procedure that may warrant INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 34 adjustment in the future. By accounting for the amount of time spent motionless or frozen as an aversive response, we may be better able to account for aversive responses that are not included in the typical gaping or avoidance behavior criteria. Additionally, it appeared as though the dual infusion training utilized in experiment two may have heightened responses towards the CS+ sucrose lever, and reduced the amount of behaviors towards the CS+ salt lever. This outcome may be due to the fact that if rats had the choice to interact with the sucrose or salt levers, they would preferentially choose to act with the appetitive sucrose lever. Accordingly, if the experimental procedure was modified to only include the independent CS+ salt and CS+ sucrose trials rather than the dual trials, rats responding towards the salt lever may become more apparent due to the fact that interaction with the sucrose lever would not be an option. One last weakness was the limited number of subjects as well as experimental materials. Specifically, the limited amount of autoshaping boxes provided that only a small number of subjects could be run at the same time. Further, given the lengthy training period, groups of animals had to be spaced weeks apart. Accordingly, given more time, a larger experimental group could be accounted for, thereby providing more significant results with reduced variance. Despite some limitations within the experimental design, there are many practical applications for the results that were obtained in the study. Knowledge of the dynamic nature of incentive salience, as well as the fact that not only can a stimulus that was never ‘liked’ become instantaneously ‘wanted’, but also that such a stimulus can become so attractive that it obtains properties of a motivational magnet, have implications for the understanding of addictive disorders such as drug addiction. By understanding that drug addiction can often present itself as a sensitization of incentive salience, and therefore manifest itself as an exaggeration of the previously mentioned behaviors, treatment and intervention methods can be modified to account INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD 35 for the qualities that come to be attributed to drug stimuli. For instance, attempting to diminish the motivational magnet properties that drug users contribute to drug-related stimuli in their environment may be a method by which desires to relapse or revert back to drug taking can be eliminated. And aside from characterizing addiction, it becomes apparent that a deeper understanding of incentive motivation for rewards has the potential to change the way we understand and treat many disorders in our society, including schizophrenia and mania. The findings of this study also have the potential to shape future research in the field of biopsychology. Seeing as the results from experiment two were inconclusive, further experimentation is required in order to measure the influence of opioid receptor activation in the CeA on dynamic incentive motivation behaviors during reward encounter. By continuing to question and test different aspects of the incentive salience hypothesis, we will not only come to deeply understand the neural networks that govern the incentive salience hypothesis, but also further enhance our understanding of reward processing in the brain. In such a way, we may eventually come to discover even more complexities in the way that the human brain processes interactions with environmental rewards, and further develop the way we study pleasure centers of the brain. INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD36 ReferencesBerridge, K. C. (2001). Reward learning: Reinforcement, incentives, and expectations. ThePsychology of Learning and Motivation, 40, 223-278.Berridge, K. C. (2009a). Wanting and liking: Observations from the neuroscience andpsychology laboratory. Inquiry, 52(4), 378-398.Berridge, K. C. (2009b). ‘Liking’ and ‘wanting’ food rewards: Brain substrates and roles ineating disorders. Physiology & Behavior, 97, 537-550. Berridge, K. C., Robinson, T. E., & Aldridge, J. W. (2009). Dissecting components of reward:‘Liking’, ‘wanting’, and learning. Current Opinion in Pharmacology, 9, 65-73.Dickinson, A., & Dawson, G. R. (1988). Motivational control of instrumental performance: Therole of prior experience of the reinforcer. The Quarterly Journal of ExperimentalPsychology, 40B(2), 113-134.Hsaio, S., & Isaacson, R. L. (1971). Learning of food and water positions by hippocampusdamaged rats. Physiology and Behavior, 6, 81-83.Krieckhaus, E. E., & Wolf, G. (1968). Acquisition of sodium by rats: Interaction of innatemechanisms and latent learning. Journal of Comparative and Physiological Psychology,65(2), 197-201. Mahler, S. V., & Berridge, K. C. (2009). Which cue to “want?” Central amygdala opioidactivation enhances and focuses incentive salience on a prepotent reward cue. TheJournal of Neuroscience, 29(20), 6500-6513.Simon, J. J., Walther, S., Fiebach, C. J., Friederich, H., Stippich, C., Weisbrod, M., & Kaiser, S.(2010). Neural reward processing is modulated by approachand avoidance-relatedpersonality traits. NeuroImage, 49(10), 1868-1874. INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD37 Stouffer, E. M., & White, N. M. (2005). A latent cue preference based on sodium depletion inrats. Learning and Memory, 12, 549-552.Stouffer, E. M., & White, N. M. (2007). Roles of learning and motivation in preference behavior:Mediation by entorhinal cortex, dorsal and ventral hippocampus. Hippocampus, 17, 147-160.Tindell, A. J., Smith, K. S., Berridge, K.C., & Aldridge, J. W. (2009). Dynamic computation ofincentive salience: “Wanting” what was never “liked”. The Journal of Neuroscience,29(39), 12220-12228.Zhang, J., Berridge, K. C., Tindell, A. J., Smith, K. S., & Aldridge, J. W. (2009). A neuralcomputational model of incentive salience. PLoS Computational Biology, 5(7), 1-14. INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD38 Author NoteMary E. Larijani, Department of Psychology, University of Michigan, Ann Arbor. I would first and foremost like to thank my mentor, Dr. Kent Berridge, for allowing me tobe a part of this exciting project. Dr. Berridge was an extremely supportive figure throughout theformulation of my thesis, and taught me a great deal about the effective ways to conduct andarticulate my research. I would also like to thank Dr. Mike Robinson for the many hours spentguiding me in my experiments and writing style. Without the helpful suggestions and insight, I would not have been able to produce a successful thesis. Thanks to all the other graduatestudents in the lab who assisted me on all other occasions. And a special thank you to myfamily: Mom, Baba, and Meena, I will always appreciate the moral support you provided mewith throughout this process. INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD39 Figure 1. Experiments one and two. Sodium appetite, measured in mL of NaCl solutionconsumed on baseline and sodium depletion testing days.* represents significant difference, p<0.05 INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD40 Figure 2. Experiment one. Appetitive behaviors towards CS+ sucrose levers in extinctionand reinforced trials during pre-­‐CS+ and CS+ presentations.* represents significant difference, p<0.05 INVESTIGATING THE DYNAMIC PROPERTIES OF REWARD41